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CTOS Course Objectives 
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Distributed Real-Time Issues 
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pplication: collection of communicating CTOS tasks on one or 
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Data of any size 
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Tasks are Connected by Symbolic Names 
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CTOS Application Configuration File 
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Format of CTOS Task Function 
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Example CTOS Task Function 
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(void *)&value, /* message data */ 
sizeof (value) , /* size of data */ 
MF_WAIT_RCVR) ; /* request a reply */ 





The msgSend() Function 
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The msgReply() Function 
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CTOS protects against incorrect replies 

— msgReply () returns ERROR if no reply was expected. 

— msgDefaultProcO will generate “missing” reply 



Reply-Wait Mechanism 
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CTOS Message Passing 









Reply-Wait Routing for Remote Tasks 





CTOS Message Passing 




Uses of Reply-Wait Mechanism 
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Example RPC Function 


o 


<D 


II 

II 


0) 


c n & 
M 0 
sc o 

H X* 
I v-/ 

W <H 
U O 
M (1) 
> N 
Cd *H 
U co 


Z’— N 




1 


m 


cd 




P 

<P 

1 - 


p> 




<D 

rH 

0 «J 


cd 




> 

P 

CO P 






P 

CO 

S «* 


1 




<D 

0) 

XJ 


a> 




CO 

p 

-» 1 

SS 

a 




z 


T3 0> 


0 


\ 



p 

•h a 

cd 

CQ 


* 


XJ 

p 

H O 

Q 





•H 

p 

>> CO 

S 

<D 




H 


a <« 

CO 

r—i 


P 


>» 

0 


1 

& 

• •* 

0 


a 

p 

~ * 

H 

P 

O 

•H 




T3 

n 

O 





«a 

•H TJ 

<*: 

TJ 

ll 

td 




H *H 




N 


<D 

0) 

P O 

1 

** 

XJ 

•H 



bO 


El« 

XJ 

•H 

rH 


or 

cd 

CO w 

S 

•H 

H 

cd 


'd 

CO 



H 


•H 


•H 

CO 

TJ 



> 

-P 


H •« 

0 

p 



CO 

•H 


bO /’■n 

a 

<D 




P 


CO PC 


CO 


W 


•rH 

0 

a 0 

-p 

TJ 


a, 

w 



cd 

CO 

rH 


>-• 

CU 

Q 

n 

11 cd 

0> 

•H 


E- 

Sx 

PI 

n 

Id 

p 

P 


1 

H 

H 



cr 

ffl 


O 

1 


XJ 

•H 

0 

bO 


M 

Q 

P 

•H 

H PS 

p 

CO 


H 

l— l 

a> 

E- 

h U 


a 



H 

> 

|H 

> 0 

T) 

V-z' 




P 

> 

m p 

P 



O 

O 

a) 

CO 

<D 

<D 

P 


Ph 

•H 

C/3 



CO 

P 


cd 

P 





P 


>> 

rtJ 





■P 


S 

P 

* 

*H 


* 

0) 



CO 


•H 


N 

P 


p 









PS 

•H 




CTOS Message Passing 



The msgBroadcast() Function 
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The msgBroadcast() Function, Con’t 
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Can not reply with data to a broadcast message 

— send a normal message to the broadcast originator 

— nzack returned by msgBroadcastO tells how many to expect 



Routing of Broadcast Message 

(VME Chassis) 
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When Can Message Data Be Deallocated? 



Message Data Management 
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args recSvr DISPLAY = mars: 0.0 

args recSvr RECLOPTIONS = -name farmAnimals -geom 600x500 
args recSvr RECLOPTIONS += -rv -iconic -fn 10x20 
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Application-wide Config-file commands 
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The 'task' command 
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<chassis> <CPU> task <symbolic-name> <funct ion-name > <priority> 

Uses a previously loaded event-handler function. 



More CPU-specific Config-file commands 
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METHOD Is Monolithic configuration file. 

% File: clifDemo.cfg 

% 

chassis vxO 1 
chassis eaxth 
sequencer earth 

PREFIX SequHost 0 

args recSvr REC.OPTIOHS = -rv -geometry 500x800+0 -name Clif_Demo 

args recSvr XCT0S_DISPLAY = earth: 0.0 

PREFIX earth 0 

task cl if Client Handler /usr2/testbed/exp/unix/demos/clif /clifClientUnix 
task gsmServer /usr2/testbed/exp/unix/bin/sun4/gsmServer 

systask viewer REVR00TDIR/unix/bin/sun4/xctosParent 

args viewer XCT0S_PR0G = REVR00TDIR/unix/bin/sun4/fxctosviewer 

args viewer XCT0S_DISPLAY = DISPLAY 

PREFIX vxO 

-1 share /usr2/testbed/exp/vxworks/share/680x0/mcsShare .o 
-1 load /usr2/testbed/exp/vxworks/lib/680x0/mcsLib . o 

0 load /usr2/t estbed/ exp/ vxworks/lib/ 680x0/ clif ManagerLib . o 

0 task clilManager clifMsgHandler 90 

1 load /usr2/testbed/exp/vxworks/lib/680x0/gripLib.o 
1 task gripL gripEvent Handler 150 

1 task gripR gripEventHandler 150 

1 load /usr2/testbed/exp/vxworks/lib/680x0/tgen.o 

1 task tgen tgenMsgHandler 100 

2 load /usr2/t estbed/exp/vxworks/lib/ 680x0/pumaLib . o 
2 load /usr2/testbed/exp/vxworks/lib/680x0/chanPuma.o 

2 task chanLPmaDrv chanLPumaHsgHandler 100 

3 load /usr2/testbed/exp/vxworks/lib/680x0/pumaLib.o 
3 load /usr2/testbed/exp/vxworks/lib/680x0/chanPuma. o 
3 task chanRPmaDrv chanRPumaMsgHandler 100 
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METHOD 2: Subordinate chassis-config files. 

% File: clifDemo.cig 
*/• 

chassis vxO 1 /usr2/testbed/exp/vxworks/demos/clif/clil! .mcs 
chassis earth 
sequencer earth 

PREFIX SequHost 0 

args recSvr REC_0PTI0NS = -rv -geometry 500x800+0 -name Clif_Demo 

args recSvr XCT0S_DISPLAY = earth: 0.0 

PREFIX earth 0 

task clifClientHandler /usr2/testbed/exp/unix/demos/clif /clirfClientUnix 

task gsmServer /usr2/testbed/exp/unix/bin/sun4/gsmServer 

5i File: clif.mcs 

% 

PREFIX vxO 

-1 share /usr2/testbed/exp/vxworks/share/680x0/mcsShare . o 
-1 load /usr2/testbed/exp/vxworks/lib/680x0/mcsLib.o 

0 load /usr2/testbed/exp/vxworks/lib/680x0/clifManagerLib.o 
0 task clilManager clif HsgHandler 90 

PREFIX vxO 1 

load /usr2/testbed/exp/vxworks/lib/680x0/gripLib . o 
task gripL gripEvent Handler 150 
task gripR gripEventHandler 150 

PREFIX vxO 1 

load /usr2/testbed/exp/vxworks/lib/680x0/tgen. o 
task tgen tgenMsgHandler 100 

PREFIX vxO 2 

load /usr2/testbed/exp/vxworks/lib/680x0/pumaLib.o 
load /usr2/testbed/exp/vxworks/lib/680x0/chanPuma. o 
task chanLPmaDrv chanLPumaLMsgHandler 100 

PREFIX vxO 3 

load /usr2/testbed/ exp/vxworks/lib/680x0/pumaLib . o 
load /usr2/testbed/exp/vxworks/lib/680x0/chanPuma . o 
task chanRPmaDrv chanRPumaMsgHandler 100 
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METHOD 3: Subordinate config files, using include 
*/, File: clilDemo.clg 

X 

chassis vxO 1 
chassis earth 

sequencer earth 

PREFIX SequHost 0 

args recSvr REC_0PTI0HS = -rv -geometry 500x800+0 -name Clif.Demo 

args recSvr XCT0S .DISPLAY = earth: 0.0 

PREFIX earth 0 

task clif Client Handler /usr2/testbed/exp/unix/demos/clif / clilClientUnix 
task gsmServer /usr2/testbed/exp/unix/bin/sun4/gsmServer 

PREFIX vxO 

-1 include /usr2/testbed/exp/vxworks/demos/clif/clif .mcs 
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METHOD 4: Subordinate chassis-config files and CVCS. 

% File: Imakefile on unix side. 

X 

LDLIBS += -lrec -lmsg -lbts -lotos -lmcs -lclifClient 
LDLIBS += -lkntpt -Icon! ig -ItranParams -lk in -ltrans 

AllTarget (clif ClientUnix clilDemo. cfg) 

UHIXBinTarget(clifClientUnix, ) 

Coni igFileTarget (clif Demo . cfg , clif Demo . cf g_src) 

*/, File: clif Demo . cfg_src 

% 

#def ine DISPLAY earth: 0.0 
tdefine HOST earth 

/* Mote: REVR00TDIR and TARGETDIR are provided by CVCS. */ 

chassis vxO i REVROOTDIR/vxworks/demos/clif /clif ,mcs_fast 
chassis HOST 
sequencer HOST 

PREFIX SequHost 0 

args recSvr REC_0PTI0HS = -rv -geometry 500x800+0 -name Clif_Demo 

args recSvr XCTGS_DISPLAY = DISPLAY 

PREFIX HOST 0 

task clifClientHandler TARGETDIR/clifClientUnix 

task gsmServer REVR00TDIR/unix/bin/sun4/gsmServer 
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METHOD 4, continued 


•/. File: Imakelile on vxworks side. 

X 

AllTarget(clil .mcs_last) 

Coni igFileTarget (clil . clg , clil . cf g.src) 

FastLoadTarget ( clil . mcs.l ast , clil . clg , clilServer , clil . clg) 


*/, File: clil.mcs.src 

X 

PREFIX vxO 

-1 share REVR00TDIR/vxworks/share/680x0/mcsShare. o 
-1 load REVR00TDIR/vxworks/lib/680x0/mcsLib . o 

0 load REVROOTDIR/vxworks/lib/ 680x0/ clilManagerLib . o 
0 task clilManager clilMsgHandler 90 

PREFIX vxO 1 

load REVR00TDIR/vxworks/lib/680x0/ gripLib . o 
task gripL gripE vent Handler 150 
task gripR gripEvent Handler 150 

PREFIX vxO 1 

load REVROOTDIR/vxworks/lib/ 680x0/tgen . o 
task tgen tgenMsgHandler 100 

PREFIX vxO 2 

load REVROOTDIR/ vxworks/lib/ 680x0/ pumaLib . o 
load REVR00TDIR/vxworks/lib/680x0/chanPuma.o 
task chanLPmaDrv chanLPumaMsgHandler 100 

PREFIX vxO 3 

load REVR00TDIR/vxworks/lib/680x0/pumaLib.o 
load REVROOTDIR/vxworks/lib/ 680x0/ chanPuma . o 
task chanRPmaDrv chanRPumaMsgHandler 100 
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Using Task-args, cont. 

CTOS-task source code for the same example: 
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A sample request at each layer of VSS: 
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msgTypeFlagSet set TYPE field of msg.flag 
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msgReplyPush push reply onto stack 

msgVarPtrSet set pointer to saved variables 

msgVarPtrGet get pointer to saved variables 
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MISC PROCESSING 

msgAckAINIT explicit acknowledgement of AINIT message 

msgApplicationExit request shutdown of the application 
msgDefaultProc default processing for system messages 













CTOS Course Exercises 


1. Simple UNIX Event Handler Task 

• Description: Single-task application which reports CTOS phases. 

• Lessons: 

— format of event handler function 

— output to X-window 

— CTOS execution phases 

— using Imakefile & cmkmf to compile/link 

— building configuration files 

— starting CTOS application 

• Initial Setup: 

(a) Change to the directory created for your team. For example, Team 1 uses 
/usr2/testbed/course/teaml/. All code written for the exercises should 
be put in this directory. 

(b) Use epath command to set up path to ‘experimental’ directory tree. 

• Procedure: 

(a) Write a standard-format CTOS task function to print a string when it receives 
MSG.PINIT, MSG.AINIT, MSG-AEXEC, MSG.ATERM, and MSG-PTERM 
messages. 

(b) Print your message string using printf, and the same string using reclnfo. 
This will demonstrate the different output locations of these two functions. 

(c) Create an Imakefile and use cmkmf command to compile and link your task 
function. 

(d) Write a CTOS application configuration file. Include the following line to 
start an Application Executive task as part of your application: 

task AppExec /usr2/testbed/course/servers/appexec 

(e) Start an app_server with Permissions=‘ Anyone’ as shown below - replacing 
workstation-name with the one assigned by the instructors. 

app_win workstation_name -p 0 

(f) Run your application via app.bts command, and observe the printed output 
(content and location). 

• Hints/Suggestions: 

— If you want to keep code from exercises separate you can create directories 
below your ‘team’ directory, e.g. exl/, ex2/, etc.; but this is not necessary. 

— Experiment with formatting the string, e.g. could use the msgMyTaskName 
function to identify your task by name in the message string. 

— Imakefile needs only 3 statements: LDLIBS, AllTarget and UNIXBinTarget. 
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- If you share an app_server you must configure the recServer to direct output 
to your terminal, e.g.: 

prefix SequHost 0 

args recSvr XCTOS_DISPLAY = xterm8:0.0 

- When sharing a workstation with another team only the first call to app.win 
will succeed, but both teams will be able to use it. If you are unable to share 
an app_server it may not have been started with the -p 0 option. 

— If your application ends abnormally it may be necessary to use app.cleanup 
to release IPC resources used by CTOS. 

2. Multi-chassis VxWorks+UNIX Application 

• Description: Port UNIX task to VxWorks, and combine all teams’ tasks into one 
application. 

• Lessons: 

— task args for configuring a task 

— compatibility between UNIX and VxWorks 

- Imakefiles for both UNIX and VxWorks 

— multi-chassis application config files 

• Procedure: 

(a) Modify your CTOS task function from Exercise 1 to save a string sent to 
it via an args command in the chassis config file, and to display the string 
during PINIT phase. 

(b) Add the args command to your config file and test it. 

(c) Create a subdirectory for a VxWorks version of your task function (this is 
necessary because the UNIX and Vxworks Imakefiles are different). 

(d) Port your CTOS task function to run under VxWorks on the VME cage 
by isolating UNIX- or VxWorks-specific code within conditional compilation 
directives, e.g.: 

#ifdef OSJJNIX 
#include <unix_stuff .h> 

#endif 

(e) Update Imakefiles and compile/link. 

(f) As a group, build an application configuration file to start UNIX and Vx- 
Works tasks for all teams in the same CTOS application. A VME CPU will 
be assigned to each team by the instructor. 

(g) Before starting the application, rlog into your VME CPU, e.g.: 
rlogin vx2 

(h) Run the application and observe the printed output on app_servers, VME 
consoles (via rlogin), and recWindow. 
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• Hints/Suggestions: 

- Using the msgStringAssign function is the best way to parse the args string. 

— The available compiler constants are OS-UNIX and OS.VXWORKS. 

- Rather than copying code into the vxworks subdirectory consider creating a 
symbolic link to the code in the ‘team’ subdirectory so that code changes are 
immediately reflected in both locations. The symbolic link can be created 
with the following command: 

In -s ../myfcn.c 

- nnlrmf will make both your team and vxworks subdirectories if you add a 
MakeSubdirsO statement to the Imakefile in the team subdirectory. 

— Designate one team to assemble input from each team to build the group’s ap- 
plication configuration file. Put this file in directory /usr2/testbed/course/. 

- Remember to include the AppExec task and to configure the RecWindow to 
display on the workstation from which the application will be run. 

- Hung applications are reset by app.cleanup. Remember to wait for the VME 
cage to reboot. 

- You will need to rlog into the VME CPU after every reboot. 

3. Send Messages to Other Tasks 

• Description: Send & receive messages among tasks of multi-chassis application. 

• Lessons: 

- finding TIDs of other tasks 

- saving data between calls to EH function 

- building and sending messages 

- accessing data contained in messages 

• Procedure: 

(a) Teams are assigned symbolic names for their tasks based on their team num- 
ber; for example, team l’s task is named Teaml. 

(b) Modify your CTOS task function from Exercise 2 to query for the TID of 
other teams’ tasks during AINIT phase. Find the TIDs of the two teams 
numerically higher and lower that yours (i.e. team 2 finds TIDs of team 1 
and team 3). Your task should report the result to the recWindow, and save 
the TIDs for later use. 

(c) During AEXEC phase, build and send messages to the tasks queried earlier. 
In the message send the string received via the config file args command. 
Use the predefined message command of MSG_STRING. 

(d) Add processing for receipt of a MSG-STRING message. Your task should dis- 
play the received string and the TID of the source task. 
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(e) As in the previous exercise, build an application configuration file that will 
start an application containing a task from each team and the Application 
Executive. Remember to include an args command for each team’s task. 

(f) Run it. 

• Hints/Suggestions: 

— We will not use the VME cage for the remainder of the exercises so you may 
remove references to VxWorks tasks from the Imakefile and config files. 

— Keep the messages identifying execution phases if you like (may be good for 
debugging), but delete printf’s and just use reclnfo’s. 

- You can test your task’s message passing by having it send a message to itself, 
and running it as a two-task application (your task plus the usual AppExec). 

4. Develop a Server Task 

• Description: Write a server and a simple task to test it. 

• Lessons: 

- requester-server architecture 

- server message protocol 

• Procedure: 

(a) Design a server to perform a simple operation. For example, your server might 
receive two floating point numbers, multiply them together, and return the 
result. 

(b) Define a message command for the request message to the server. Offset the 
message command value from MSG-USER+100 by your team number (i.e. 
team 5 should use MSG.USER+ 100+5). 

(c) Define a data structure to contain the data transmitted in the request message 
to the server. A data structure for return data may also be needed unless the 
server returns a simple data type such as a single floating point value. 

(d) Develop a server task function which receives your request message containing 
input data, performs its operation, and returns the result in a reply message. 

(e) Write a task function that builds and sends a request message to your server 
and displays the returned results. 

(f) Compile, link and run your server and test task to ensure correct operation. 

• Hints/Suggestions: 

— Each team works alone for this exercise, but in the next exercise we will 
combine results. Keep in mind that other teams will later use your server. 

- Use of a header file is highly recommended for this exercise, and will be 
required for the next one. 

- Next exercise we will assign symbolic names to servers based on team number 
(e.g. Team3_Svr) - you might as well start using this name now. 
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- Can avoid the need of a data structure for server reply data if you return 
just a simple data type. However, you will need to specify a way to identify 
error conditions; for example by defining a certain return value to be an error 
indication. 

— Hint: you should use the reply-wait mechanism to block the test task until 
the server replies, so you will use a MF_WAIT_SENDER, .RECEIVER, or 
.COPY message flag for the request message. 

- A good server should check all inputs to ensure that they are valid. For 
instance, what happens if the received request message has a NULL msg.data 
pointer? 

— Try several requests to the server, perhaps including invalid inputs or inputs 
that result in an illegal server operation. 

5. Write a RPC Function 

• Description: Encapsulate access to your server as a remote procedure call so 

others can use it. 

• Lessons: 

— RPC functions 

- header files and libraries 

• Procedure: 

(a) Name your server from Exercise 4 based on your team number, i.e. the server 
written by team 3 will get symbolic name Team3_Svr. 

(b) Encapsulate the building of the server request message and all communication 
with the server into a RPC function, and put this function in a separate file. 
Compile it as a library (see Imakefile UNIXLibTarget () command). 

(c) Create a header file which contains only the definitions needed to access your 
RPC function (potentially as little as the function prototype). Put enough 
documentation in the header file so that users from the other teams will be 
able to understand how to use your RPC. 

(d) Build symbolic links in directory /usr2/testbed/course/servers/ to your 
RPC header and library, and to the executable code for your server. 

(e) Modify your task function from Exercise 4 to call your RPC function, and 
test it. 

(f) Experiment with calling the RPC functions developed by the other teams. 
To do so you will need to include the header file in your test task that calls 
the RPC, and to start their server task when you run the application. 

• Hints/Suggestions: 

— The server developed in Exercise 4 can probably be used without modification 
for this exercise. 
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Pick a name for your RPC function and library that will avoid conflicting with 
other teams’ names (we don’t want ten versions of my RPC). Including your 
team number in the RPC name is a good way to accomplish this (myRPC3 
is OK, although boring). 

When using other team’s RPCs you will need to include their header files and 
link with their libraries. Add the following lines to your Imakefile to enable 
cmkmf to find the RPCs: 

CPPFLAGS += -I/usr2/testbed/course/servers/ 

LDLIBS +■ -L/usr2/testbed/course/servers/ 

If the linker cannot resolve a reference to a RPC function it may be because 
the library was not named in the Imakefile LDLIBS statement. 

In a week or so the directories used during the exercises will be deleted. Copy 
any files you’d like to keep into your own account soon. 
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File: Imakefile 

Written By: Don Lefebvre 

Date: Nov 18, 1992 

Purpose: Imakefile for Exercise 




SOLUTION TO EXERCISE 2 I 
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prefix vxO 

1 chdir /usr2/testbed/course/answers/ex2/vxworks/ 

1 load ex2.o 

1 task MyVxWorksTask ex2 100 

1 args MyVxWorksTask TEXT = test of task args to VxWorks 
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if ( (t idl - msgTidQuery (myTid, "Teaml")) — 0) AllTarget (ex3) 

reclnfo (myTid, "Could not find tid of Team l\n“); 

else UNIXBinTarget (ex3, ex3.o) 

reclnfo (myTid, "Team 1 TID- lx\n", tidl); 



prefix SequHost 0 

task AppExec /usr2/testbed/course/servers/appexec 
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%l File: app.cfg 

%% Written By: Don Lefebvre 

%% Date: Nov 10, 1992 

4% Purpose: Application configuration file for Exercise 



multlplyRPCO multiplies two double-precision floating point numbers 







return (msgDefaultProc (myTld, msg>> ; I I I I I I I I I 1 1 1 1 I I I I 1 1 1 1 I I 1 1 I I I I I 1 1 1 1 1 I I 1 1 I I I I 1 1 1 1 I I I I I I I I I I 1 1 I I 1 1 I I 1 1 I I 1 1 I I I I I III 



operandl; j 

operand2; return (msgDefaultProc (myTid, msg) ) 
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